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Related Applications 

[0001] This application claims priority to U.S. Provisional Application Serial 
Number 60/421,255 entitled "Streaming of Digital Data to a Portable Device", 
filed on October 25, 2002 which is incorporated herein by reference in its entirety. 

Background 

I. 1 . Field of the Invention 

[0002] The present invention relates generally to the use of infrared and/or 
radio frequencies to stream digital data to portable devices in a mass audience or 
in designated broadcast area. 

II. 2. Description of the Background Art 

[0003] There are several circumstances which may interfere with a person's 
ability to hear, and thereby interfere with the presentation of information to the 
listener. For example, persons with hearing loss may miss narratives, sound 
effects, music and other sound material related to the presentation, and 
messages in live performances, films, television and special events. Persons 
who do not speak the language or languages used in the presentation may miss 
narratives and other related messages that are presented. The language barrier 
prevents many people from different cultures and languages from understanding, 
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participating or interacting with the infon nation being presented. Background 
environmental noise may also affect a person's ability to hear, and thereby 
diminish the effect of the presentation. Additionally, presentations often could be 
made more effective if they included a method to provide interactivity between 
the audience and the presenter(s). 

[0004] Captioning systems have been used in many venues including 
museums, theaters and other auditoriums to provide foreign language translation 
or captioning for the hearing impaired. These systems are either 1) "open 
captioning" on a projected surface or large adjacent display area where the entire 
audience can see the captioning; 2) reflective captioning using a transparent but 
reflective panel to display the text from a rear projection while allowing the viewer 
to see the display or performance through the panel; or, 3) hard-wired displays in 
the back of the seat in front of the viewer. 

[0005] Wireless streaming systems also exist. These consist of modulated 
analog and digitized audio broadcasting systems such as assistive listening 
systems and wireless headphones. Also short-range (often 1 meter or less) 
digital transceivers are typically used to exchange data between computers or 
between computers and peripherals. These systems do not synchronize multiple 
units. 

SUMMARY OF THE INVENTION 

[0006] It is therefore an object of this invention to provide a method of 
presenting random and/or synchronized information such as narratives, 
translations, interactive games, control signal commands or other show related 
messages, to patrons of shows, movie theaters, exhibit halls, auditoriums and/or 
designated areas through an unobtrusive device. 

[0007] The present invention makes use of infrared (IR) emission and/or radio 
frequency (RF) transmission to stream data to a portable device for closed 
captioning, language translation for multi-cultural language groups, previews, 
games, control of devices and/or similar applications. The portable captioning 
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device could be carried by a person and/or the device could be placed in a fixture 
for hands free use. Before or at the time of the start of a presentation or 
presentations, the IR/RF system will start the transmission of data to the portable 
units. The transmission will synchronize the portable device with the 
presentation or presentations for captioning, language translation, previews, 
games, control of devices and/or similar applications. The portable device might 
be based on existing technologies such as mobile phone, personal digital 
assistant (PDA) or a combination of both mobile phone and PDA, a custom 
designed device specifically for this application, or an interactive device. This 
system can be combined with an existing audio stream for the hearing impaired, 
descriptions for the blind and/or language translation. For example, the invention 
may provide for infrared streaming for assistive listening systems. 

[0008] Possible user device implementations include a custom portable text 
display terminal having a receiver, display controller, micro-controller, small 
amount of memory, and power source. Similarly, a custom audio unit having a 
receiver, digital-to-analog converter, audio amplifier, and speaker, could also be 
used. 

[0009] Other devices include talking or motorized toys and games capable of 
receiving content appropriate to a presentation. Such devices may also allow 
interaction with nearby units. Special effects units having some combination of 
lights, audio effects, or animations could also be used. These units could be held 
by the user or mounted in the presentation space as a remotely controlled 
device. Still other devices include a digital map or way-finding unit, messaging 
system that allows the user to broadcast and receive data to and from nearby 
units, or user identification unit that broadcasts user identity, preferences, or 
location to nearby units. 

[0010] The present invention provides several benefits over prior art systems. 
Prior art systems are not aware of user preferences, such as the user's 
language, gender, age, etc. Prior art wireless audio systems suffer from signal 
loss and deterioration of the transmitted audio when the user moves relative to 
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the infrared transmitter. Existing digital wireless audio systems do not have 
provisions for mixing text or control data in the audio data stream. 

[0011] The present invention provides for a combination of features not found 
in prior art wireless data communication systems. For example, the system and 
method of the present invention allows multiple users' devices to simultaneously 
receive identical data and remain synchronized with the presentation(s). User 
devices are able to receive different types of data, such as audio, animation 
control, text, etc. during a single transmission. User devices are furthermore able 
to interact with the presentation, interact with the presenters, or interact with each 
other. 

Brief Description of the Drawings 

[0012] Fig. 1 is a schematic view of a system of the invention utilizing IR 
signals. 

[0013] Fig. 2 is a schematic view of a system of the invention utilizing RF 
signals. 

[0014] Fig. 3 is a schematic view of a portable display device receiving an IR 
signal. 

[0015] FIG. 4 is a schematic view of an embodiment of the present invention 
utilizing radio frequency (RF) signals. 

[0016] FIG. 5 is an exemplary depiction of a data packet stream delivered by a 
high speed transmitter. 

[0017] FIG. 6 is an exemplary embodiment of the portable device in 
accordance with the present invention. 

[0018] FIGs. 7a-c is a flow diagram in accordance with the teachings of the 
present invention. 

Detailed Description of the Preferred Embodiments 

[0019] Reference will now be made in detail to an exemplary embodiment of 
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the present invention, an example which is illustrated in the accompanying 
drawings. It is to be understood that other embodiments may be utilized and 
structural and functional changes may be made without departing from the 
respective scope of the present invention. 

[0020] The hardware requirements of the preferred embodiment of this system 
are depicted in Figs. 1 and 2. A time code reader 10, which in this embodiment 
is an optical time code reader, is capable of reading time codes from a show 
device 12 such as a film projector, show control computer or another media 
source. The media source may include analog and/or digital content and could 
be audio, video, and other information (e.g., force feedback parameters for tactile 
sensing). A central processing unit (CPU) 14 receives the time code signal from 
the reader 10 and synchronizes the content with the film and/or presentation, 
which may be a movie screen 15. Examples of content are text captioning, 
language translation, games and/or other related applications. The central 
processing unit 14 has the capability to access and interpret the content. The 
content can reside in the internal memory of the central processing unit 14 and/or 
as a removable memory media. 

[0021] The CPU 14 will deliver the synchronized data to the infrared (IR) 
emitters 16, which are capable of delivering IR messages or control data to a 
portable device 18, which may be operated either in an indoor or outdoor 
environment. Alternatively, a low-powered licensed and/or non-licensed radio 
frequency (RF) system can also be used to deliver the synchronized data to the 
portable device via an RF signal. Fig. 2 depicts an RF receiver 24 and RF 
transmitter 22, which interact with a portable RF capable device 18 to achieve 
the same results as the IR system previously described. 

[0022] The portable display device 18 has sufficient memory to allow storage of 
all data to be presented. All required data may be transmitted to the device 18 in 
one session for display at predetermined times after receipt of a start signal, or 
the data may be transmitted in a plurality of sessions for display in real time. The 
stored data may be displayed in its entirety, or a portion of the data may be 
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displayed. The device 18 includes infrared ports capable of receiving and/or 
emitting infrared messages. Messages emitted from the portable display device 
18 are read by one or more IR detectors 19. In an alternative embodiment, the 
device 18 could also contain RF receiver and/or transmitter ports capable of 
receiving and transmitting RF messages. The portable device 18 will receive the 
IR or RF signal and convert the signal to information that can be stored and/or 
displayed in sync with the presentation. The device 18 may also contain the 
capability to receive and play audio such as for assistive listening and/or audio 
language translations, or program material specific to the presentation. 

[0023] The system could also recognize show/presentation start and/or end 
signals. The system could then transmit random and/or synchronized 
information to the patrons possessing a device. This will allow the patrons to 
interact with the device while waiting for the show/presentation to start or after 
the show/presentation has ended. 

[0024] One possible application of the invention is depicted in Fig. 3. In this 
application, time code information is read by the CPU 14, which then accesses 
on board content, and delivers the synchronized data to the infrared emitters 16, 
which are in turn capable of delivering IR messages. A portable device 24 
receives the IR messages and converts the IR messages to presentable data. In 
this instance, the IR receiver 19 is a modification to an existing device such as a 
PDA (for example, a Palm type device) and/or a pocket PC (for example, a 
Compaq iPAQ) that can store and/or immediately display the data. The IR 
receiver takes the IR signal from the emitter and translates it to an electronic 
signal for the serial port 22 of the PDA and/or pocket PC. A terminal software 
program converts the electronic signal into data that is presented as text on the 
display screen 26. 

[0025] This invention may be adapted for uses such as text captioning and 
language translation in movie or live theaters, consumer products which can 
provide an interactive experience, and to provide a wireless link for control 
signals to equipment, devices or products which are used in public presentations. 
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[0026] Another application of the invention is depicted in Fig. 4. In this 
application, a combination of RF transmitter 30 and IR emitters 31 broadcast 
data to portable devices 33. The portable devices have both RF and IR 
receivers. The RF data stream contains encrypted caption text, digitized audio, 
and control data for one theater 34 or multiple venues. Each theater or venue 
has one or more IR emitters that send small amounts of data to synchronize the 
portable device playback to user position or the theater presentation. An emitter 
mounted outside one theater's entrance may send an identifying code that the 
portable device uses to identify that theater. The device then extracts data from 
the RF stream for that theater's content, and loads the content into the device's 
memory. Content may include theater name, show times, show synopses, as 
well as the text and audio for an upcoming show. When the user enters the 
theater, the device receives IR synchronizing codes for the current show, and the 
device displays caption text, or play audio, or otherwise synchronize device 
functions during the show. The IR synchronizing codes are derived from the 
show media as described earlier. 

[0027] FIG. 5 illustrates an exemplary embodiment of the basic structure of the 
data packet stream sent by the high-speed transmitters. 

[0028] The figure shows a possible broadcast packet stream of theater show 
content for three theaters, along with data packets of ancillary information such 
as theater show schedules and current date and time. 

[0029] The broadcast packet stream contains small groups of data (packets) 
that contain a small amount of content that will be received by the user devices 
and, if needed, will be loaded into the user devices local memory. The logical 
state of each user device determines which packets are loaded into a particular 
user device. For example, user devices that are in theater A would receive and 
store content for theater A, but ignore content for theaters B and C. All user 
devices would receive the current time packet, and use the time data to reset the 
user device clock. Any user device in front of theater A, B, or C would receive 
and store the theaters' schedule data. The example packets in the figure show 



7 



Express Mail Label No. EV 3 1 1273502 US 



Docket #: 54317-022500 



content broken into one-second fragments. So a captioning device in theater A 
would receive the caption text packet for the 1st second of the show, caption text 
packet for the 2nd second, etc. Each second of content would be loaded into 
device memory for access during the show. 

[0030] A large number of individual packet types are possible, each containing 
different data such as caption text, audio content, video content, graphics, 
images, time, schedules, menus, geographical information, game content, survey 
questions, advertising, warnings, alerts, etc. 

[0031] Each data packet (refer to examples Caption Text, Audio Content, 
Current Time Packets in FIG. 5) always starts with a unique Header value that 
signifies the start of a new data packet. 

[0032] The data packet also contains a Count value that is used to indicate the 
amount of following data in the packet. The Count is used to validate the packet 
and calculate a checksum. 

[0033] The Encryption data represents key and authentication information. 
This, and the Checksum, enables the user device to verify that following data in 
the packet is intended for the device and that the data has not been corrupted. 
Corresponding key and authentication information are loaded into the user 
device. 

[0034] The Message Type value defines the content data in the packet. Based 
on the Message Type, the user device will determine how to decode the packet 
data, where to store the data, how and when to use the data. For example, 
warning text may be immediately displayed; show caption text may be buffered in 
device memory until an IR synchronizing code triggers that caption text display. 
Menu or schedule information may be buffered in device memory until the user 
requests it. 

[0035] The Address value designates an identifier for a receiving device. This 
may be a unique address, so the packet is intended for only one user device. Or, 
the address may specify a block of user devices, such as all captioning devices, 
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or all game devices. Or, the Address may denote the packet as a generic 
broadcast sent to all devices. 

[0036] The optional Start Time value acts as a packet sequence number, but 
may also be used by the receiving user device to define when the packet content 
is used. As packets are received, the user device will buffer content into its 
memory in sequence based on start time. As the user device receives 
synchronizing codes, it will pull content from the sequential memory or search 
memory for the content with appropriate start time. Additional information such as 
Caption Text Data with Formatting codes, Digitized Audio content, and Current 
Time and Frame information are included in the Data Packet Stream. 

[0037] there is a Checksum information that uses a combination of all the 
preceding data to verify that the packet has arrived at its location in its entirety. 

[0038] FIG. 6 illustrates an exemplary embodiment of a portable user device in 
accordance with the present invention. All portable devices have the common 
elements of an IR receiver 50, RF receiver 60, IR transmitter 51, RF transmitter 
61, processor 52, memory 54, and power source 56. Other elements of the 
devices vary depending on intended applications. In this exemplary embodiment, 
the portable device has both a display 53 for displaying text and/or graphics, in 
addition to an audio amplifier 57 for playing sounds, which may be in the form of 
speech or musical tones. Memory 54 is where the data is stored on the device. 
The IR receiver 50 receives a code from the wireless transmitter, and processor 
52 analyzes the code. The processor 52 interprets the code to determine how to 
handle it. If the code is intended for that device, the device searches its memory 
54 for content corresponding to the code. The content is then displayed on 
display 53 and/or plays sound through the audio amplifier 57. The portable 
device shown in FIG. 5 also shows user inputs 55, such as pushbuttons, which 
may be used for interactive applications. The high-speed RF transceiver system 
could be used to transfer content to the portable device, while the IR tranceiver 
system may be used to precisely aim location synchronizing codes to the 
portable device. 
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[0039] FIG. 7 is a flowchart that shows how received data packets are handled 
by the user device. After the device is powered on, it is capable of receiving data 
packets from both IR and RF receivers. The device also monitors user inputs 
that may come from pushbuttons, touch screen, speech input, assistive devices, 
etc. The device also periodically reads its clock, and searches its event database 
memory for preloaded events set to trigger at the current time. 

[0040] Either receiver may recognize an incoming packet. Any packet that 
contains timing errors, corrupted data, cannot be authenticated, etc. will be 
ignored. A packet that has correct bit timing, correct header, can be 
authenticated, can be decrypted, and has a correct checksum will be marked as 
valid. 

[0041] A valid packet will be compared with the user device configuration, so 
that only appropriate packets will be used. For example, a user device that 
displays caption text, but has no audio capability, may be configured for text only. 
That device would ignore packets containing audio content. 

[0042] The packet message type determines how the device will use the packet 
data. The packet may contain content such as caption text, audio, or video that 
is buffered in the user device content database memory for future use. Other 
data, such as an urgent text alert message, may be sent to the device display 
immediately. Data such as location and time synchronizing codes may cause the 
device to search its content database memory, extract the content, and present 
the content. Current time data may cause the device to reset its internal clock. 
Event time data will be placed in the device's content database memory, and will 
be acted on when the internal clock matches the event time. Special effect data 
may cause the user device to buzz, ring, animate, etc. 

[0043] The user device may have some number of user inputs available. 
These may take the form of pushbuttons, touch-screen soft buttons, touch- 
screen menus, speech input, assistive device such as a breath-operated puff 
switch, etc. The user may set preferences, such as points of interest, types of 
movies, age, educational level, etc. The device may use these manually entered 
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preferences, along with learned preferences, to alter the presentation of content. 
For example, a child's device may be set to display captions using a small 
English vocabulary. As the child carries the device from location to location in a 
museum, the device may also "learn", or infer, the child's preference for a certain 
type of display. 

[0044] The device may also learn a user's direction of travel, and therefore 
predict or suggest future destinations. For example, after walking down a hallway 
towards North, the device may alert the user that the hallway will end, and a turn 
to the East or West is necessary. Furthermore, the learned direction, along with 
any learned and manually entered user preferences, may cause the device to 
deduce a preference for an exhibit that is to the East of the hallway end. 

[0045] The portable device may automatically infer user preferences from use 
of the device based on the current location of the user device, the time elapsed 
at a location, path history (i.e. geographic walking history) of locations visited, 
and the time elapsed between locations (i.e. walking speed). 

[0046] User inputs may also be assigned to device hardware functions. For 
example, an input pushbutton may cause an IR transmitter to send a code to an 
adjacent device. Such code may identify the user to another user, or identify the 
user to an automated device, network, etc. One code may cause a door to open, 
for example, or allow interaction with a game system, etc. 

[0047] The user device may present content based on internal logic and its 
internal clock, without receiving data packets. For example, text captions may 
appear at a predefined time of day to remind a user of venue show times, closing 
time, or remind the user of medication needed, etc. Further, such time events 
may be logically constrained by the learned and manually entered user 
preferences. For example, a restaurant closing time may not be displayed 
because the user has walked away from the restaurant. 

[0048] In closing it is to be understood that the embodiments of the invention 
disclosed herein are illustrative of the principals of the invention. Other 
modifications may be employed which are within the scope of the invention. 
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Accordingly, the present invention is not limited to that precisely as is shown and 
described in the present specification. 
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